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Summary 

Purpose 

: —  The  purpose  of  the  present  study  was  to  facilitate  understanding  of  the 
nature  of  mental  disorder  diagnoses  by  examining  the  extent  to  which  the 
degree  of  diagnostic  specificity  (i.e.,  group,  type,  or  subtype)  and  the 
social  context  (i.e.,  certain  circumstances  under  which  diagnoses  take  place) 
affect  diagnostic  agreement  (the  reliability  of  measures  across  time).  Also, 
the  present  study  investigated  the  transformative  nature  of  certain  mental 
disorder  diagnoses  (i.e.,  how  some  mental  disorder  diagnoses  change  by  their 
very  nature)  by  tracking  classifications  both  into  and  out  of  particular 
diagnostic  categories. 


The  approach  of  this  study  was  to  analyze  data  from  the  Navy  Enlisted 
Career/Medical  History  File.  The  population  consisted  of  all  hospitalized 
cases  of  active  duty,  enlisted  Navy  personnel  between  1981  and  1984,  inclu¬ 
sive,  with  a  mental  problem  as  the  primary  diagnosis.  Two  nonexclusive 
subsamples  were  investigated.  The  first  subsample  (N=2,132)  consisted  of 
hospitalized  cases  which  were  later  reviewed  by  a  U.S.  Navy  physical  Eval¬ 
uation  (P.E.)  Board.  The  second  subsample  (N=5,40?)  consisted  of  cases 
involving  multiple  hospital  admissions  for  a  mental  disorder. 

Findings 

Diagnostic  group  (i.e.,  psychotic  versus  nonpsychotic)  was  a  more  reliable 
measure  than  diagnostic,  type  (e.g.,  schizophrenia  versus  personality  disor¬ 
der).  Diagnostic  type,  in  turn,  was  a  more  reliable  measure  than  diagnostic 
subtype  (e.g.,  chronic  catatonic  schizophrenic).  And,  certain  diagnostic 
types  and  subtypes  were  consistently  more  reliable  across  time  than  others. 
Except  for  alcoholism  and  personality  disorders,  the  Kappa  values  associated 
with  diagnostic  types  and  subtypes  in  the  P.E.  Board  subsample  were  signifi¬ 
cantly  larger  than  those  in  the  multiple  hospitalization  subsample. 

Conclusions 

The  present  study  found  that  degree  of  diagnostic  specificity  and  the 
social  context  affected  the  reliability  across  time  of  ICD-9  mental  disorder 
diagnoses.  Also,  the  nature  of  certain  mental  disorders  (i.e.,  disorders 
which  progressively  deteriorate,  those  which  resemble  one  ..mother  because  of 
shared  features,  and  those  whicli  occur  in  combination  with  other  disorders) 
affects  diagnostic  reliability. 
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Introduction 

In  a  recent  article  in  Science,  Faust  and  Ziskin  (July,  1988)  argued  cor¬ 
rectly  that  many  studies  have  demonstrated  the  poor  reliability  of  specific 
mental  disorder  diagnoses,  but  then  also  created  the  false  impression  that: 
1)  all  mental  disorder  diagnoses  are  similarly  unreliable,  and  2)  the  inac¬ 
curacy  of  diverse  forms  of  clinical  judgement,  as  indicated  by  the  inability 
of  clinicians  to  achieve  diagnostic  agreement,  to  predict  violence,  feigned 
behavior,  or  brain  damage,  is  attributable  to  a  common  set  of  limitations 
(i.e.,  the  same  factors  explain  problems  in  very  different  and  complex  areas 
of  clinical  judgement).  Spitzer,  Williams,  and  Pincus  ( Science ,  November, 
1938)  responded  by  saying  that  psychiatry  has  recognized  the  problem,  is  doing 
something  about  it,  and  that  the  "rest  of  medicine  also  has  problems  with 
reliability."  Joseph  Matarazzo  (quoted  by  John  Bales,  Monitor ,  January, 
1989),  the  current  President  of  the  American  Psychological  Association, 
responded  by  saying  that  Faust  and  Ziskin' s  (1988)  conclusions  were  based  on  a 
narrow  review  of  the  literature  (i.e.,  primarily  negative  studies)  and  ignored 
other  studies  which  do  not  suffer  from  methodological  flaws.  However,  neither 
the  critique  by  Faust  and  Ziskin  (1988),  the  defense  by  Spitzer  et  al .  (1988), 
nor  the  corrective  comments  uy  Matarazzo  (Bales,  1989)  provide  a  clear  agenda 
for  understanding  the  nature  of  mental  disorder  diagnoses,  or  suggest  ways  to 
improve  diagnostic  reliability. 

The  purpose  of  the  present  study  was  to  facilitate  under s t and i ng  of  the 
nature  ol  mental  disorder  diagnoses  by  examining  the  extent  to  which  the 
degree  of  diagnostic  specificity  (i.e.,  group,  type,  or  subtype)  and  the 
social  context  (i.e.,  the  different  conditions  under  which  diagnoses  take 
place)  affects  diagnostic  agreement  (i.e..  the  reliability  of  measures  across 
time).  Also,  the  present  study  in  'est iga ted  the  turns  formative  nature  of 
certain  mental  disorder  diagnoses  (i.e..  lmw  some  diagnoses  may  change  by 
their  very  nature)  by  tracking  classifications  both  into  and  out  or  pattieulat 
diagnostic  categories. 
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Methods 


Subjects 

The  population  consisted  of  all  hospitalized  cases  of  active  duty,  en¬ 
listed  Navy  personnel  between  1981  and  1984,  inclusive,  with  a  mental  problem 
as  the  primary  diagnosis.  Two  non-exclusive  subsamples  of  this  population 
were  investigated,  which  corresponded  to  different  social  contexts.  The  iiist 
social  context  subsampie  (N=2,132)  consisted  of  hospitalized  cases  which  veie 
later  reviewed  by  a  U.S.  Navy  Physical  Evaluation  (P.E.)  Board.  The  mean 
length  of  time  between  hospitalization  and  last  P.E.  Board  review  was  206.74 
days  (sd=214.45).  The  second  social  context  subsample  (N=5,402)  consisted  of 
cases  involving  multiple  hospital  admissions  for  a  mental  disorder  ■  The  mean 
It  rgths  of  time  between  hospitalizations  were:  1)  143.55  days  (sd=225.08) 
between  first  and  second  hospitalization,  2)  lib. 82  days  (sd=187.59)  between 
second  and  third  hospitalization,  3)  91.00  days  (sd=149.42)  between  third  and 
fourth  hospitalization,  4)  91.45  days  (sd=139.05)  between  fourth  and  fifth 
hospitalization,  and  5)  176.78  days  (sd-251.13)  between  f^rst  and  last  hospi¬ 
talization. 

Procedures 

Data  collection  Procedures .  Data  were  obtained  from  the  Navy  Enlisted 
Career/Medical  History  File  (NECMHF).  NECMHF  is  based  on  two  compiled  files. 
One  is  the  Service  History  File,  which  consists  of  demographic  and  military- 
service  history  data  from  Navy  Military  Personnel  Command  in  Arlington,  Vir¬ 
ginia.  The  other  is  the  Medical  History  File,  which  contains  hospitalization, 
death,  Medical  Board  action,  and  Physical  Evaluation  Board  action  data  from 
Naval  Medical  Data  Services  Cen  v  in  Bethesda,  Maryland.  NECMHF  is  compiled 
and  maintained  by  the  Naval  Health  Research  Center,  San  Diego.  California 
(Garland,  Helmkamp,  Gunderson,  Gorham,  Miller,  McNally,  &  Thompson,  1987). 

Coding  Procedures .  Primary  mental  disorders  were  based  on  ICD-9  codes  and 
were  coded  to  correspond  to  three  levels  oi  diagnostic  specificity.  The  three 
levels,  which  are  henceforth  referred  to  as  group,  type,  and  subtype,  corre¬ 
spond,  respectively,  to  low,  moderate,  and  high  degrees  o£  diagnostic  specifi¬ 
city.  At  the  lowest  level  of  diagnostic  specificity,  tire  group  level,  all 
mental  disorder  diagnoses  were  relegated  to  either  the  psychotic  (all  psycho¬ 
tic  diagnoses  combined)  or  nonpsychot it-  (all  nonpsychotic  diagnoses  combined) 
groups.  At  the  moderate  level  of  specilici tv.  the  type  level,  the  range  of 
possible  mental  disorder  diagnoses  included  the  following  diagnostic  types: 
1)  organic  psychoses,  2)  schizophrenia,  3)  affective  psychoses,  4)  para 
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noia,  5)  other  psychoses,  6)  unspecified  psychoses,  7)  neurotic  disorders, 
8)  personality  disorders,  9)  alcoholism,  10)  transient  situational  distur¬ 
bance,  and  11)  drunkenness.  At  the  highest  level  of  specificity,  the  subtype 

level,  each  type  of  mental  disorder  diagnosis  was  further  classified  into  its 

specific  subtypes  (e.g.,  chronic  catatonic  schizophrenic).  In  both  subsani 
pies,  five  contrasts  involving  the  three  levels  of  diagnostic  specificity 
(group,  type,  and  subtype)  were  computed  to  assess  diagnostic  reliability 
across  time:  1)  psychotic  group  versus  nonpsyrhotic  group  at  lime  1  was 

compared  to  psychotic  group  versus  nonpsychotic  group  at  Time  2  (2  x  2  Table), 
2)  all  types  of  mental  disorders  at  Time  1  were  compared  to  all  types  of  men¬ 
tal  disorders  at  Time  2,  3)  all  subtypes  of  mental  disorders  at  Time  1  were 
compared  to  all  subtypes  of  mental  disorders  at  Time  2,  4)  each  type  of  men¬ 

tal  disorder  versus  all  other  types  combined  at  Time  1  was  compared,  respec¬ 
tively.  to  its  corresponding  type  of  mental  disorder  versus  all  other  types 
combined  at  Time  2  (2  x  2  Tables),  and  5)  each  subtype  of  mental  disorder 

versus  all  other  subtypes  combined  at  Time  1  was  compared,  respectively,  to 

its  corresponding  subtype  of  mental  disorder  versus  all  other  subtypes  com¬ 
bined  at  Time  2  (2  x  2  Tables). 


Results 

The  degree  to  which  diagnosis  at  Time  1  agreed  with  diagnosis  at  Time  2 
was  assessed  using  the  Kappa  statistic.  Kappa  assesses  the  chance-corrected 
consistency  (or  reliability)  of  a  set  of  measurements  across  time.  Kappa 
equals  zero  when  obtained  agreement  equals  chance  agreement.  Greater  than 
chance  agreement  leads  to  positive  Kappa  values,  while  less  than  chance  agree¬ 
ment  leads  ro  negative  Kappa  values.  With  negative  values,  the  degree  of 
agreement  has  little  practical  importance  (Cohen,  1960).  Table  1  shows  the 
Kappa  values  for  different  levels  of  diagnostic  specificity  across  multiple 
hospital  admissions  and  at  time  of  hospital  ization  versus  time  ot  last  P.E. 
Board  review. 

Two  sets  of  analyses  were  conducted.  The  first  set  of  analyses  combined 
the  P.E.  Board  subsample  with  the  multiple  hospital  admissions  subsample  in 
order  to  assess  the  effect  of  diagnostic  specificity  on  reliability  across 
time.  The  Vilcoxon  Signed-Ranks  Test  was  used  to  compare  the  Kappa  values  for 
different  levels  of  diagnostic  specificity.  Tire  Kappa  values  comparing  psy¬ 
chotic  group  versus  nonpsychotic  group  at  Times  1  and  2  were  significantly 
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larger  than  the  Kappa  values  comparing:  1)  all  types  of  diagnoses  at  Time  1 
versus  all  types  of  diagnoses  at  Time  2,  and  2}  all  subtypes  of  diagnoses  at 
Time  1  versus  all  subtypes  of  diagnoses  at  Time  2  (both  Z  values=-2 . 20,  p 
values i two-tailed ]< .05) .  The  Kappa  va’  les  comparing  all  types  of  diagnoses  at 
Time  1  veLsus  all  types  of  diagnoses  at  Time  2  were  significantly  larger  than 
those  comparing  all  subtypes  of  diagnoses  at  Time  1  versus  all  subtypes  of 
diagnoses  at  Time  2  tZ=-2.20,  p[two-tailed]<.05) .  The  Kappa  values  (Tables  2 
and  3)  for  each  respective  diagnostic  type  versus  all  other  types  at  Times  1 
and  2  were  significantly  larger  than  those  for  each  respective  diagnostic  sub- 
type  versus  all  other  subtypes  at  Times  1  and  2  (Z—5.C3,  p[ two-tailed] <. C01 ) . 

The  second  set  ol  analyses  combined  the  diagnostic  types  and  subtypes  in 
the  ?.E.  Board  subsample,  and  combined  the  diagnostic  types  and  subtypes  in 
the  multiple  hospital  admissions  subsample.  The  Wilcoxon  Signed-RaiiKs  Tes1' 
was  used  to  compare  the  combined  Kappa  values  ol  the  two  different  subsamples 
in  order  to  assess  the  effect  of  the  social  context  on  reliabili  .y  across 
time.  In  subsample  1,  Kappa  values  were  computed  to  measure  reliability  at 
time  of  hospitalization  versus  time  of  last  P.E.  Board  review.  In  subsample 
2,  Kappa  values  were  computed  to  measure  reliability  at  time  of  first  hospi¬ 
tal  admission  versus  time  of  last  hospital  admission.  The  Kappa  values  of  the 
two  subsamples  (types  and  subtypes  combined,  respectively)  did  not  differ 
significantly  (p=.16).  However,  two  supplemental  analyses  were  conducted  to 
assess  the  possibility  that  opposing  directional  effects  in  the  two  subsamples 
masked  a  social  context  effect.  The  first  supplemental  analysis  consisted  of 
only  the  types  and  subtypes  with  larger  P.E.  Board  subsample  Kappa  values  (12 
of  the  16  ranks  or  all  types  and  subtypes  except  alcoholism  and  personality 
disorders)  and  found  a  significant  difference  between  the  two  subsamples 
(Z=-3.06,  p| two-tailed ]  <.01).  The  second  supplemental  analysis  consisted  of 
only  alcoholism  and  personality  disordei  types  and  subtypes  (the  remaining  4 
ranks)  and  found  a  significant  difference  between  the  two  subsamples  ( Z = - 1 . 8 3 , 
p ( one- t ai led ]  s.Ob).  Except  for  alcoholism  and  personality  disorders,  Kappa 
values  were  significantly  larger  in  the  P.E.  Board  subsample  than  in  ti.~  mul¬ 
tiple  hospitalization  subsample.  The  Kappa  values  for  alcoholism  and  person¬ 
ality  disordei  types  and  subtypes  were  significantly  larger  in  the  multiple 
hospitalization  subsample  than  in  the  P.E.  Boaid  subsample.  Thus.  lie  impact 
of  the  social  context  on  diagnostic  reliability  depended  on  the  puiti'ulai 
diagnostic  type  or  subtype. 
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Tablo  1 


Reliability  (Kappa)  Across  rise  of  Hants!  Diagnosis  by  Laval  of  Specificity 


Number  of  Hospital  Admissions _  Tima  of  Hospi- 


Level  of  specificity 

1  vs  2 

2  vs  3 

3  vs  4 

4  vs  b 

1st  vs  Last 

Kow 

Me  a  i . 

talication  vs 
Last  PE  Board 

Group  vs.  Group 

.  5f 

.  57 

.53 

.67 

.  54 

.  57 

.55 

Type  vs .  Type 

.  36 

.  37 

.43 

.44 

.  3  ) 

.  39 

.52 

Subtype  vs .  Subtype 

.01 

.  14 

.02 

.03 

.  1  3 

,  07 

.01 

Table  2 

Rel  i  rbiJ  ity  (Kappa)  Across  Ti»®  of  Rental  Diagnosis  Type; 


Time  of  Hospi- 

Number  of  Hospital  Admissions _  talization  vs. 


Type  vs.  Else 

1  vs  2 

2  vs  3 

3  vs  4  4 

vs  5 

1st  vr.  Last 

Last  FE  Board 

Organic  Psychoses 

.15 

.  06 

.16 

.  15 

.13 

.31 

Sch i zophren ia 

.5? 

.61 

.03 

.73 

.58 

.72 

Affective  Tsivhcres 

.42 

-  57 

.  5  4 

.  59 

.  3  8 

6 1 

Other  Psychoses 

.25 

.  30 

.16 

.66 

.21 

.35 

Neu*  oses 

.26 

.12 

.11 

.03 

.22 

.  54 

Personality  Disorders 

.95 

.  36 

.41 

.40 

.32 

.1* 

Alcohol  ism 

.52 

.51 

.47 

.58 

.50 

.14 

Transient  Situational  Disturbance 

.  24 

.  22 

.39 

.  26 

.20 

.33 

Drunkenness 

.10 

.10 

.19 

.  26 

.09 

-- 

Table  3 

Reliability  (Kappa)  Across  Yi*e  of  Rental  Diagnosis  Subtype 


i 


Number  of  Hospital  Admissions 


Time  of  Hospi¬ 
talization  vs. 


Subtype  vr.  . 

Else 

1  vs  2 

2  vs  3 

3  vs  4 

4  vs  5 

1st  VS  Last 

Last  PE 

Organic  Fsychoses 

(14 

subtypes ) 

.14 

.07 

.07 

.09 

.  16 

.23 

Sch i zoph i en i a 

l  1  1 

subtypes ) 

.44 

.  4<> 

.  -J  7 

.  5  4 

.41 

.  55 

Affective  Fsychoses 

(9 

subtypes ) 

.  29 

.  36 

.  29 

.41 

.  2  6 

.40 

Other  Psychoses 

(1? 

subtypes  ! 

.19 

.23 

-  la7 

.66 

.16 

.  3  2 

Neut oses 

(8 

subtypes ) 

.23 

.13 

.  1  3 

-.0  2 

.  1  9 

.  50 

Personality  Disorders 

(9 

subtypes ) 

.29 

.32 

.44 

-  *10 

.26 

.15 

Ale  :>ho i  i  r.m 

1  6 

subtypes ) 

.42 

.42 

.  4  t 

.  50 

.39 

.08 

Transient  Situational 

Distut bance 
(  1  2  subtype*-.  ) 

.  10 

.10 

.  19 

.  2  6 

.  OQ 

.21 

Drunkenness 

i 

subtype  1 

.  17 

.  16 

.32 

.  1  ? 

.15 

— 

Boa  t  d 
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The  pattern  of  diagnostic  change  was  then  assessed  by  examining  classifi¬ 
cations  across  time  both  into  and  out  of  a  given  type  of  mental  disorder.  The 
pattern  of  diagnostic  change  of  primary  mental  disorder  diagnoses  was  differ¬ 
ent  in  the  two  subsamples  of  the  study.  Tables  4-5  show  the  shift  in  diagno¬ 
sis  from  time  of  hospitalization  to  time  of  last  P.E.  Board  review.  Tables 
6-7  show  the  shift  in  diagnosis  from  time  of  first  hospital  admission  to  time 
of  last  hospital  admission. 

In  the  P.E.  Board  subsample .  diagnostic  change  appeared  tu  be  due  t.o  the 
problem  of  differential  diagnosis  of  psychotic  mental  disorders  (the  failure 
to  distinguish  between  different  mental  disorders  which  share  particular  psy¬ 
chotic  symptoms  but  which  differ  in  the  overall  pattern  of  psychotic  symptom¬ 
atology)  and  the  progressive  deterioration  of  some  nonpsycho  tic  mental  disor¬ 
ders.  Kor  example,  Table  4  clearly  shows  that  at  time  of  last  I'.F.  Board 
review,  four  of  the  six  psychotic  mental  disorders  (#1,4, 5, 6)  were  not  well 
differentiated  from  schizophrenia  and  that  all  of  the  nonpsychot ic  disorders 
(#7-11)  indicated  some  deterioration  into  a  more  severe  mental  (psychotic) 
disorder.  Table  5  clearly  shows  a  consistent  trend  toward  classification  into 
a  new  category  of  diagnosis  at  time  of  last  P.E.  Board  review  but  cannot  dis 
tinguisb  between  problems  of  differential  diagnosis  and  progressive  deter iora 
t  ion . 

In  the  multiple  admission  subsample,  on  the  other  hand,  diagnostic  change 
was  more  complex  and  appeared  to  be  due  to:  1)  the  problem  of  differential 

diagnosis  foi  both  p^jchotic  and  nonnsychotic  disorders,  2)  t lie  problem  of 
multiple  diagnosis  (clusters  of  disorders  |e.g.,  certain  personality  disorders 
and  substance  abuse  disorders]  which  are  oftentimes  related  and  which  may 
alternate  as  the  principal  [primary]  diagnosis),  3)  and  the  progressive 
deterioration  of  some  nonpsychot i o  disorders.  For  example,  Table  6  shows 
clearly  t ha t  schizophrenic  and/or  personality  disorders  were  frequently  con 
fused  with  all  seven  psychotic  mental  disoiders  (#1-7).  Table  6  also  shows 
substantial  diagnostic  change  among  t he  nonpsychot ic  disorders  that  could  be 
attributable  to  progressive  deterioration,  differential  diagnosis,  or  multiple 
diagnosis.  The  pattern  of  diagnostic  change  for  drunkenness  (#11),  improper 
use  of  drugs  (#13),  and  nervousness/debility  (#17)  looks  like  movement  tovaid 
a  worsening  of  symptoms.  The  pat  rein  of  change  lot  neuroses  (#B),  dtug  depMi 
dence  (#12),  physical  disotdei  of  p-  y<  Imccn  i  ■  origin  (#14).  ^pec-  ■)  vymp',.m 
(#15),  and  transient  situational  d  ist.  in  ban,  c  (itlw)  could  hr  a  i  t  i  i  bu :  a  l>  I  <  '■> 
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Clanflii  icstion  Out  of  *  bia^nontio  Type 
at  Tino  of  La nt  I'.li.  Hoard 


Hospitalisation  Diagnosis  _ r.h.  Board  Pi  ft  gnoBis 


1 1 ) 

Psychosis  aysociat  with  other 

V:»y«.hosis  associated  with 

physical  conditions  tn=24' 

othai  physical  conditions 

17% 

Schi sophrenla 

46% 

Affective  psychosis 

25% 

(  2  ' 

Schi zopht ani a  (n«777) 

Schi 7oph  r  an l a 

947 

Affective  psychosis 

3% 

(  3  ) 

Affective  psychosis 

Affect iv*  psychosis 

84% 

Schirophrem  a 

G% 

Nervousness /deb i 1 i t  y 

4% 

(  4  ) 

Faranoid  (ti  =  46) 

Paranoid 

4  1  * 

Schifcophi on . a 

2ri» 

(5) 

Other  psychosis  (n  =  f>7) 

Other  piychoMS 

Schi soph i eni a 

(  6  ) 

Unspecified  psychosis  (n*93) 

Unspecified  psychosis 

2** 

Schuoplrema 

(7) 

Nourosas  <n=150) 

Neuioms 

Sch l reph i »n ja 

1  07 

Affective  psychosis 

81 

(  8  ) 

Personality  disorders 

Personality  disorders 

i  n 

Schizophrenia 

?:% 

Affective  psychosis 

1  3% 

Nouioses 

117 

( 9  j 

Alcoholism  (n-90! 

Alcohol  ism 

3  t 

Scliisoph .  en  i  a 

4.C  N 

Affectiv.  psychosis 

3  1  7 

(  10  ) 

Transient  situational 

T •  ansi  ent  •> i  t  u a  t  r  o n .1  1 

2  S  * 

disturbance  (n-135) 

d  i  S  t  Ul  bailee 

Affect i vo  psychosis 

2  1% 

Schizof  hr  am  a 

in 

Nuwioses 

io\ 

m ) 

Nei vousnts s/deb i 1 i t  y  (n  =  129) 

Nor  vcmsnesb  'debility 

271 

Attec.tivo  psychosis 

Naur oses 

1  1 1 

Tabl«  5 


Classification  Tn*o  a 
at  Time  of  Last 

I’O'ipiiSiisAUon  in  ignos {  s 

Diagnostic  Type 

P  .  R .  Board 

r.L.  Hoard  Diagnosis 

!  V  ) 

Schi zophrenir 

74% 

Tchi  zoph  i  en  i  a  m-984> 

Unspecified  p: 

s  % 

r«i  a one lity  d\f. ocdei" 

5  ' 

Other  psychos  ir. 

n 

Affective  psychosis 

3% 

(2> 

Affective  psychosis 

59% 

Affective  psychosis  (n»502) 

Transient  situational 

t-% 

disturbance 

Fersonality  disordtis 

6% 

Alcohol  i  r.m 

6% 

(  3  ) 

Ta  t  anoi d 

4*>% 

Paranoid  ( n»4 1 ) 

Fersonality  dicotdi'.-s 

1  3% 

<«> 

Other  psychosis 

33% 

Othat  psychosis  (n»52) 

Personality  disorders 

12% 

Schizophrenia 

8  % 

Unspecified  psychosis 

43% 

Unspecified  psychosis  (n*56 ) 

Othar  psychosis 

11% 

Schizophrenia 

11% 

( t>  1 

N*Ul  C6«5 

54% 

Neuroses  { 1  / 1  ) 

Transiant  situational 

15% 

distui bancs 

Na  t vousness/debi lity 

11% 

reiscnality  disorders 

1  * 

C  ) 

Personality  disorders 

39% 

Personality  disorder?.  tn«52) 

Affa*"tive  psychosis 

1  b% 

Pa  t  anoid 

13% 

^ch i zoph  k  ania 

12% 

(8) 

Physical  disorder  of 

57% 

Physical  disorder  cf 

psychogenic  origin 

paychogonic  origin  (n=23) 

Neuroses 

30% 

(  9  1 

Special  symptoms,  not 

78% 

Special  symptoms,  not 

elsewhere  classified 

elsewnere  classified  (n«4l. 

Alcohol ism 

10% 

Te iso na lity  disorders 

7% 

(  10) 

Transient  situational 

60% 

Transient  si trational 

disturbance 

disturbance  ( n - 57 ) 

Nervousness/dobility 

18% 

111) 

Ne  r  vousnes  c  /  del'i  1 1 1  y 

37% 

Ne  t  vousnes  b/d«’b  '  lity  (  n»95  ) 

Transient  situational 

16% 

disturbance 

Affective  psychosis 

15% 

Neuroses 

13% 
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Table  fe 


Clasui f irat i on  oat  of  •  Diagnostic  Type 
at  Yin«  of  bast  Hospital inat i on 


Fust  Hospitalisation  L'locnosi'*. 


Last  Hospi  t  a  1  i  r.a  1 r  on  Diaqnopi 


( 1  > 

Alcoholic  psychosis  (n«6b) 

Alcoholism 

74% 

Schizophrenia 

9% 

( 7 ) 

Psychosis  associated  with  other 

Psychosis  associated  wit  It 

physical  conditions  ( n  -  3  v  ) 

ot  lie  i  physical  conditions 

14% 

Sch  i  zoph  i  «n  i  a 

32% 

A co ho  1  \  s m 

1  6  1 

l % 1 

?•  hizophi  <tnia  ln-747) 

Sell  1  ?Opl)  I  61)  1  l 

7  2% 

J  e i a  o  n  a  1 1 1  y  disorders 

l ;% 

(  4  ) 

Affective  psychos it-  m~122j 

Affective  psychosis 

4  3% 

Personality  disorders 

1  9% 

(S) 

Paranoid  (n^?l  ' 

rata no i d 

1  4% 

Sch  i  ?.  oph  i  en  i  a 

331 

personality  disorders 

291 

( «** ) 

Other  psychosis  (ir=7-l' 

OUtet  psychosis 

141 

T-f  h  i  r.oph  i  «n  i  a 

26% 

Personality  disorders 

22% 

(7) 

Unspecified  psychosis  (n  78) 

Unspecified  psycho? is 

9% 

Jch i fcopn i en i a 

11% 

Per  r.ona  1  \  ty  disorders 

27% 

Affective  psychosis 

10% 

(8) 

Neuroses  (n=273 ) 

Neu  toco  r- 

2  2% 

le i sonil j  ty  disorders 

29  % 

Alcohol ism 

i  ?  % 

Transient  situational  distnih. 

1  3% 

l  9  ) 

petsonility  disorders  m- ?•■**') 

le  i  r;ona  .  i  t.y  di  solders 

T'7* 

A1  or  h  >1  i  •* »;i 

1  3% 

Transient  situational  dir. Tut  L. 

7% 

Sch  i  r.oph i  oni  a 

f  1 

(  10  I 

Alcoholism  (n  =  l,M7) 

A 1  coho 1 1 sm 

7  29 

P  e  i  s  c*  r  a  1 1 1  y  d  i  z  o  r  rl  e  r  s 

ft% 

Drunkenness 

S  t 

T  i  n  r.  s  i  e  n  t  situational  drat  t  h . 

l>  % 

(11)  Pi  in; kann ln=311, 


(12)  Drug  dependence  ( n  =  b  4  ) 


(1*)  Improper  use  of  dtug*  (n=29) 


l  1  4  )  1  h y  s  i f: a  1  (irjoidei  of 

psychogenic  origin  f  n  -  2  7  • 


(lb)  special  i  ympt  ojn^ .  not  else 

Wheie  classified  (NF.C)  :n  2ft) 

(16)  Transient  situational  distux 
banco  (n-52b) 


(17)  Nor  voi* iner-s/dobi  1  i ty  ( n  —  1 1 4  l 


Pi  unkeiiness 

A]  O'ho  1  i  s m 

1'ei  ion*  lily  d  i  *  i  d  a  i  n 

Drug  dependence 
Alcohol  ism 

Personality  disorders 

Imp  lope  i  u  ■.*'  of  dings 
Alcohol  j  nm 

Per  p.om  1 1  ty  riir.oui»'i  <; 

Physical  d? solder  vf 
psychogenic  nil  gin 
Personality  di  so  r  doi  *. 
Alcohol i sm 

Special  symptoms.  NEC 
Personality  disorder;: 


\4 


Transient  situational 

disturb. 

23% 

Pei  r-onal  i  t  y  J  i  s  •-»  r  de  r  r. 

3b% 

■ 

Al cohol i sm 

18% 

.  i1. 

Neu  l  oser. 

7% 

} 

Ne  t  vousnes s/debi 1 i t  y 

20% 

t;  ■  j 

Personality  disorder* 

26% 

!  "1  '■ 

Transient  situational 

disturb. 

!»•% 

1  $ 

Alcohol i sm 

13% 

< 

n 


Table  1 


Classification  Into  a  Diagnostic  Type 
ot  Tina  of  Last  Hospitalisation 


Fust  Hospitalization  Diagnosis 


Last  Hospitalisation  Diagnosis 


(It  Alcoholic  psychosis  13% 

Alcoholism  67% 

>  2  '  Fsychesis  associated  with  21% 

other  physical  conditions 
Alcoholism  25% 

Drug  dependence  17% 

(3‘  Schizophi enia  531 

Personality  disorders  10% 

Unspecified  psychosis  9% 

Other  psychosis  6% 

(4l  Affective  psychosis  37% 

Transient  situational  disturb.  11% 
Personality  disorders  10% 

Tier  vousness/debiliiy  10% 

(51  ether  psychosis  22% 

Personality  disordei©  28% 

Alcoholism  13% 


Alcoholic  psychosis  (n*»46) 


psychosis  associated  with  other 
physical  conditions  (n*24) 


f»ch  i  r.opht  em  a  (n*3  36> 


Affective  psychosis  (n*144) 


Other  psychosis  (n~46) 


(6  ) 

Unspecified  psychosis 

20% 

Unspecified  psychosis 

(n=35> 

Alcoholism 

1-1* 

Other  psychosis 

11* 

5ch i zophreni a 

11% 

O) 

Neuroses 

32% 

Neuroses  (n*I85) 

Transient  situational  disturb. 

21% 

.Alcohol  i  5!T 

m 

Personality  disorders 

n% 

(  8  ) 

Feisonality  disorders 

35V 

Personality  disorders 

ln=895] 

Transient  situational  disturb. 

21% 

Alcoholism 

14% 

Neuroses 

9% 

(9) 

Alcohol  ism 

6  7% 

Alcoholism  (n«*l,64  3) 

Drunkenness 

13% 

Tiansient  situational  disturb. 

6* 

Personality  disorders 

4  % 

(10  ' 

Prunkennes  s 

21%. 

Drunkenness  (n=157) 

Alcohol  ism 

50% 

Personality  disorders 

9% 

Transient  situations!  disturb. 

7%. 

ill' 

Drug  dependence 

30% 

Drug  dependence  (n«47) 

Alcohol  ism 

26% 

(  12  , 

Improper  use  of  drugs 

10% 

Improper  use  of  drugs 

(  n=-4  0  ) 

Alcoholism 

30% 

Drug  dependence 

10% 

<  n 

Physical  disorder  of 

38% 

Physical  disorder  of 

psychogenic  origin 

psy  hogenic  origin 

ln=2l ) 

Nsut oses 

33% 

l  14  » 

Speci&l  symptoms,  not. 

65% 

Special  symptoms,  NEC 

( n  =  2  3 ) 

elsewhere  classified  (NEC) 

Neut  oses 

13% 

Personality  disorders 

13% 

(  15  i 

Transient  situational  disturb. 

3’% 

Transient  situational 

Al  coho  1 isn 

22% 

disturbance  (n=323) 

Personality  disorders 

1  1% 

Neut osen 

11% 

(16  ] 

Ne r vousnes s/debi 1 i t y 

26% 

Nervousnes  s/debi 1 i t  y 

(n-91  ) 

Transient  situational  disturb 

.  20% 

Personality  disorders 

12% 

Neuroses 

12% 
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eithei  differential  diagnosis  or  multiple  diagnoses.  The  consistent  associa¬ 
tion  of  personality  disorders  (#9)  and  alcoholism  (ttlO)  suggests  multiple 
diagnoses . 

Discussion 

The  present  study  found  that  degree  of  diagnostic  specificity  and  tin- 
social  context  affected  the  reliability  across  time  of  ICD-9  mental  disorder- 
diagnoses.  Diagnostic  group  (i.e.,  psychotic  versus  nonpsycho t i c)  was  a  more- 
reliable  measure  than  diagnostic  type  (e.g.,  schizophrenic  disorder  or  person¬ 
ality  disorder).  Diagnostic  type,  in  turn,  was  a  more  reliable  meastue  than 
diagnostic  subtype  (e.g.,  chronic  catatonic  schizophrenic).  And,  certain 
diagnostic  types  and  subtypes  (e.g.,  schizophrenic  and  chronic  catatonic 
schizophrenic,  respectively)  were  consistently  more  reliable  across  time  than 
others  (e.g.,  personality  disorder  and  antisocial  personality  disorder, 
respectively).  Except  for  alcoholism  and  personality  disorders,  the  Kappa 
values  associated  with  diagnostic  types  and  subtypes  in  the  P.E.  Board  subsam¬ 
ple  (time  of  hospitalization  versus  time  of  last  P.E.  B  aid  review)  were  sig¬ 
nificantly  larger  than  those  in  the  multiple  hospitalization  subsample  (i.e., 
time  of  first  hospital  admission  versus  time  of  last  hospital  admission). 
That  social  context  effect  was  attributable  to  the  fact  that  Navy  P.E.  Boards 
screen  for  more  severe  disorders  as  a  primary  basis  for  service  discharge 
(Kilbourne,  Hilton,  and  Goodman,  1988).  Moderate  to  large  Kappa  values  in  the 
P.E.  Board  subsample  indicated  relative  stability  in  tire  severity  of  a  given 
mental  disorder  (especially  for  psychotic  types  and  subtypes),  while  lower 
Kappa  values  generally  indicated  a  shift  from  a  less  severe  mental  disorder  ro 
a  more  severe  mental  disorder.  For  example,  alcoholism  is  treated  on  eithei 
an  outpatient  01  inpatient  basis  in  the  U - S -  Navy  (e.g.,  there  aie  treatment 
programs  lasting  for  six  weeks  in  many  Navy  hospitals,  which  increases  the 
likelihood  of  diagnostic  agreement  while  in  the  hospital)  and  is  not  routinely 
used  as  the  basis  for  service  discharge  (which  decreases  the  likelihood  of 
diagnostic  agreement  if  an  individual  initially  hospitalized  for  alcoholism  is 
latei  reviewed  by  a  Navy  P.E.  Board). 

Thus,  the  nature  of  certain  mental  disorders  (e.g.,  disorders  which  pin 
gressively  deteriorate,  those  which  resemble  one  another  because  oi  shared 
features,  and  those  which  occm  in  <umli  i  na  •  1  on  'it.h  othei  disorders)  affect: 
diagnostic  reliability.  Diagnostic  change  and  uncertainty  (i.e.,  symptom 
change  and  symptom  overlap)  are  an  inherent  aspect  oi  some  mental  disorders. 
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It  may  only  be  after  a  battery  of  psychological  tests  has  been  administered  o: 
treatment  has  begun  that  a  final  diagnosis  can  be  specified  with  some  degree 
of  confidence.  Even  then,  the  clinician  might  have  to  concede  that  the  final 
diagnosis  is  not  the  only  possible  diagnosis  hut  the  most  plausible  wot  king 
diagnosis.  Notwithstanding,  while  problems  with  differential  and  multipl< 
diagnosis  aic  two  ateas  that  clinicians  need  to  work  on  to  improve  diagnosti< 
reliability  (two  areas  that  are  emphasized  in  the  revised  third  -edi t ion  of  the 
Diagnostic  and  Statistical  Manual  of  Mental  Disorders  (DSM-JIl-R,  1987 j),  it 
is  also  evident  that  diagnostic  change  can  be  affected  by  certain  organiza¬ 
tional  settings  and  policies. 

Taken  together,  these  findings  suggest  that  in  actual  practice  a  "close 
enough  principle"  steers  most  clinical  diagnoses,  and  is  probably  the  general 
rule  regardless  of  the  classi f icat ion  system  or  range  of  diagnostic  types  exa¬ 
mined.  That  observation  is  by  no  means  alarming  since  effective  treatment  is 
not  contingent  upon  diagnostic  exactness.  Ihe  matching  of  specific  psycho- 
therapies  or  specific  medications  to  specific  mental  disorders  is  not  critical 
for  the  successful  treatment  of  most  mental  disorders.  Treatment  success  has 
resulted,  in  large  part,  from  the  nonspecific  eliects  of  therapy  (e.g.,  expec¬ 
tations  of  help  and  hope,  new  learning  experiences ,  successful  outcomes,  and 
increased  self-mastery  [Frank,  1974;  Sloan,  Stables,  (,'ristol,  Yorkston,  & 
Whipple,  1975;  Smith  f.  Glass,  1977;  Stiles,  Shapiro,  &  Elliot,  1986])  and/or 
the  general  effects  associated  with  a  particular  class  of  medication  (e.g., 
neuroleptics,  antidepressants,  or  sedatives/hypnotics).  An  individual  suffer¬ 
ing  from  a  particular  mental  disorder  may  respond  positively  to  any  one  of 
many  different  forms  of  psychotherapeutic  intervention  (e.g.,  psychodynamir , 
cognitive-behavioral,  humanistic,  etc.). 

When  specificity  of  treatment  (e.g.,  behavioral  oi  medical)  has  been  indi¬ 
cated  by  the  weight  of  empirical  evidence,  such  specificity  is  really  only 
approximate  in  application.  For  example,  theie  are  several  techniques  of.  the 
behavioral  approach  or  different  forms  of  a  particular  class  of  medication 
whicli  produce  similar  effects,  and  the  same  technique  or  form  of  medication  is 
not  always  administered  in  The  same  way.  More  importantly,  such  approximate 
specificity  in  application  does  not  hold  true  foi  all  people  at  all  times  with 
a  particular  mental  disorder. 

Thus,  for  all  practical  purposes,  the  belief  rliat  one  has  hi'  the  dingim- 
tic  "bull's  eye"  may  impose  limitations  upun  the  full  i  ange  of  psy.  h<>  t hoi  ap<-u 
tic  intervention.  On  the  other  hand,  the  recognition  that  mental  diagnoses 
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are  inexart,  but  close  enough,  assures  appropriate  boundaries  around  the  pro¬ 
blem  (c.g.,  the  focus  of  treatment)  as  well  as  sufficient  flexibility,  given 
cttl  turn)  and  individual  differences,  in  Heating  the  problem.  The  acceptable 
lange  oi  diagnostic  closeness  is  an  empiiical  question  that  remains  to  be 
de  t  ei mi ned . 

Hie  therapeutic  alliance  (the  special  relationship  that  develops  between 
the  therapist,  and  the  client  l Frank,  1974;  Strupp,  1986;  Kilbourne  &  Richard¬ 
son,  1988])  is  mote  important  in  the  long  run  tor  helping  most  individuals 
deal  with  their  psychological  problems  than  either  diagnostic  or  treatment 
exactness.  It  is  within  the  context  of  the  therapeutic  alliance  and  certain 
common  clinical  strategies  (e.g..  corrective  expei iences  and  feedback  [Gold- 
fried,  1 980 ] )  that  an  individual  learns  new  ways  to  tackle  unmet  needs  and  un¬ 
resolved  conflicts,  distorted  perceptions,  irrational  beliefs,  negative  affec¬ 
tive  states,  faulty  communication  patterns,  inappropriate  behaviors,  unreward¬ 
ing  relationships,  and  to  acquire  a  positive  sense  of  self. 
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Foot  notes 
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